Multi-level Data Fusion for the Detection of Targets using multi-spectral Image Sequences

This paper presents an approach to the long range automatic detection of vehicles, using multi-sensor image sequences. The method was tested on a database of multi-spectral image sequences, acquired under diverse operational conditions. The approach consists of two parts. The rst part uses a semi-supervised approach, based on texture parameters, for detecting stationary targets. For each type of sensor one learning image is chosen. Texture parameters are calculated at each pixel of the learning images and combined using logistic regression into a value that represents the conditional probability that the pixel belongs to a target given the texture parameters. The actual detection algorithm applies the same combination to the texture features calculated on the remainder of the database (test images). When the results of this feature level fusion are stored as an image, the local maxima correspond to likely target positions. These feature-level-fused images are calculated for each sensor. In a sensor fusion step the results obtained per sensor are then combined again. Region growing around the local maxima is then used to detect the targets. The second part of the algorithm searches for moving targets. In order to detect moving vehicles, any motion of the sensor needs to be detected rst. If sensor motion is detected, it is estimated using a Markov Random Field model. Available prior knowledge about the sensor motion is used to simplify the motion estimation. The estimate is used to warp past images onto the current one in a temporal fusion approach and moving targets are detected by thresholding the diierence between the original and warped images. Decision level fusion combines the results from both parts of the algorithm.